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SIMULATION  WITH  MARKOV 
TRANSITION  MATRIX  MODEL  OF 
A  REQUISITION  PROCESSING  SYSTEM 

IRWIN  F.  GOODMAN 

ABSTRACT 

A  cursory  review  of  the  literature  relating  to  the 
application  of  the  Markov  Transition  Probability  Matrix  for 
the  evaluation  and  analysis  of  problems  was  accomplished. 

A  FORTRAN  IV  computer  time  sharing  program,  based  upon  the 
mathematics  of  Markov  transition  Matrices,  has  been 
developed  and  documented,  The  program  was  initially 
developed  with  data  based  upon  a  classical  random  walk 
problem  involving  a  drunk  meandering  from  corner  to  comer 
between  his  home  and  a  bar.  The  resulting  Markov  Model 
has  been  applied  to  a  requisitioning  system,  an  essentially 
equivalent  problem.  Some  analysis  results  are  presented 
following  the  application  of  the  computer  program  to  a 
requisitioning  system.  The  computer  program  has  been 
written  generally  enough  for  application  to  such  other 
diverse  problem  areas  as  charge  accounts,  tank  battles 
and  reliability  and  maintainability. 
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A.  Introduction 

A  cursory  review  of  the  literature  related  to  Markov 
transition  probability  matrix  modeling  was  accomplished. 

A  short  bibliography  of  related  books  is  included  in  a 
later  section.  Some  of  the  more  essential  Markov  tech- 
nicques  of  analysis  were  organized  together  and  then 
programed  in  FORTRAN  for  solution  on  a  computer  time 
sharing  terminal.  The  purpose  of  the  model  is  to  describe 
a  requisition  processing  system.  An  equivalent  problem 
involves  the  classical  random  walk  problem  involving  a 

drunk  meandering  from  corner  to  corner  between  his  home  and 

( 

a  bar.  Therefore,  initially,  the  computer  program  and 

\ 

model  were  developed  and  checked  out  based  upon  the 
random  walk  problem.  Then,  the  model  was  applied  to  a 
requisitioning  system. 

The  data  used  in  this  report  are  presented  only  for\ 
the  sake  of  illustrating  the  theory  employed.  The 
findings  are  consistent  with  the  data  presented  but 
therefore  not  necessarily  with  the  real-world  situation. 


B.  Markov  Chains 


A  Markov  Chain  is  defined  as  a  probabilistic  process 
in  which  the  probability  of  moving  from  ore  state  to  another 
state  may  depend  on  the  present  state,  but  on  no  other  past 
history.  Classically,  this  process  is  usually  examplified 
in  terms  of  the  "wandering  drunk"  problem  which  is  an 
example  of  a  random  walk. 

"Wandering  Drunk"  Problem 

A  long  street  has  eight  intersections.  A  drunk  wanders 
along  the  street,  His  home  is  located  at  intersection  1 
and  hia  favorite  bar  at  intersection  number  8. 

-/ - / - / - / - / - / - / - /- 

12345678 

HOME  BAR 

Intersections  2-7  Are  Referred  to  As  Street  corners 

At  each  intersection  other  than  his  home  or  the  bar,  he 
moves  in  the  direction  of  the  bar  with  probability  1/4  and 
in  the  direction  of  his  home  with  probability  3/4.  He 
never  wanders  down  a  side  street.  If  he  arrives  at  his  home 
or  the  bar,  he  remains  there.  When  he  remains,  we  say  that 
the  process  .8  "absorbed". 
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Some  of  the  typical  questions  that  an  analysis  of  this 
proDiem  would  address  itself  to  are  as  follows: 

a.  What  is  the  chance  that,  starting  at  a  given 
corner,  the  drunk  will  end  up  at  his  home  or  at  the  bar? 

b.  If  the  drunk  starts  at  a  particular  corner,  how 
many  blocks,  on  the  average,  will  the  drunk  walk  before  being 
"absorbed",  that  is,  arrive  at  his  home  or  the  bar? 

Stated  in  the  Markov  language  of  probability: 

What  are  the  absorption  probabilities,  and  what  is  the 
mean  time  to  absorption? 

The  transition  probability  matrix  and  diagram  for  this 
problem  are  as  follows: 


-  •  i 

TRANSITION 
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TRANSITION  PROBABILITY  DIAGRAM 
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In  summary,  the  home  and  bar  states  are  referred  to  as 
absorbing  states  and  the  other  states,  CORNR  2  thru  CORNR 
7,  are  referred  to  as  transient  states.  In  the  case  of 
absorption  states,  once  there,  the  probability  of  remaining 
there  is  one  (certainty) .  On  the  other  hand  with  regard  to 
the  transition  states,  it  is  possible  to  leave  as  well  as 


enter  them. 
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The  transition  probability  matrix  for  the  "wandering 
urunk;i  problem  was  processed  by  the  MARKl  computer  program. 
The  computer  print  -  out  is  included  in  a  later  section. 

The  results  are  briefly  discussed  as  follows:  Assuming  the 
drunk  initially  can  be  at  any  one  of  the  corners  2  thru  7 
(away  from  the  home  or  bar)  with  equal  probability  1/6,  then 
in  the  long  run  he  will  end  up  at  his  home  with  probability 
.92  and  end  up  at  the  bar  with  probability  .08.  With 
regard  to  the  typical  questions  discussed  earlier,  the 
results  are  as  follows: 

The  probability  that  starting  at  a  given  corner,  the 
drunk  will  end  up  at  his  home  or  the  bar  are  as  follows; 


STATE 

HCME 

BAR 

CORNR  2 

.999 

.001 

CORNR  3 

.996 

.004 

CORNR  4 

.988 

.012 

CORNR  5 

.963 

.037 

CORNR  6 

.889 

.  Ill 

CORNR  7 

.667 

.  333 

If  the  drunk  starts 

at  a 

given  corner, 

the  quantity  of 

blocks,  on  the  average, 

that 

he  will  walk 

before  arriving 

at  his  home  or  the  bar  are  as  follows: 
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C-  "A  Requisition  Processing  System"  Problem 

Markov  Transition  Probability  Matrix  reasoning  and 
techniques  of  analysis  which  were  briefly  discussed  above 
in  terms  of  a  classical  random  walk  problem  have  been 
applied  to  a  requisition  processing  system.  The  system 
has  been  greatly  simplified  here  for  the  sake  of  presenta¬ 
tion  of  ideas  and  the  mathematical  modeling.  The  system 
is  primarily  automatic,  consisting  of  a  high  spaed 
automatic  data  processing  computer  system  and  the  logic 
of  a  highly  sophisticated  inventory  control  system  which 
together  process  the  transactions.  A  requisition  represents 
a  demand  on  the  system  by  a  customer  for  materiel.  This 
type  of  transaction  (requisition)  and  the  many  other 
transactions  necessary  to  keep  the  records  current  and 
decisions  reliable  are  processed  by  the  computer  inventory 
control  system. 

A  successful  attempt  has  been  made  to  mathematically 
model  some  of  the  aspects  of  such  a  system.  To  begin  with, 
only  the  requisition  type  transactions  were  studied.  Such 
a  transaction  was  considered  to  be  in  the  following  states: 
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ncwxuiio  Lwiu^i.ci.cu  ; 


MRO  —  Mafpripl  Rplpaqo  Order 
PASORD  -  Passing  Order 
REJCUS  -  Reject  to  Customer 
BACKOP,  -  Back-Order 
Actions  to  be  completed: 

FRSPAS  -  First-Pans 
SUBPAS  -  Subsequent  -  Pass 

CQMPLT  -  Completed  action  is  defined  as  any  one 
of  the  above  four  states  (MRO,  PASORD, 
REJCUS,  BACKOR) 

Two  versions  were  modeled,  one  for  all  the  above  states, 
four  absorbing  and  two  transient,  and  the  other  one  con¬ 
taining  only  one  absorbing  state  (in  place  of  the  four 
absorbing  states)  and  two  transient  states  (same  two 
transient  states) .  The  models  are  referred  to  as  Model  A 
and  Model  B.  The  states  considered  for  each  model  are 
shown  below: 
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MODEL  B 


nxjus^u  M 

Absorption  States;  Absorption  States: 

MRO  COMPLT 

PASORD 

REJCUS 

BACKOR 

Transient  States:  Trainsient  States: 

FRSPAS  frspas 

SUBPAS  SUBPAS 

The  Model  B  version  was  defined  with  less  states  to 
facilitate  a  more  in-depth  variation  of  parameter  study. 
The  results  of  the  variation  of  parameter  study  are 
presented  in  a  later  section.  The  transition  probability 
matrices  and  diagrams  for  the  Model  A  and  Model  B  versions 
of  the  requisition  processing  system  are  as  follows: 
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MODEL  A 


TRANSITION  PROBABILITY  DIAGRAM 
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MODEL  B 


TRANSITION  PROBABILITY  DIAGRAM 
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The  earlier  questions  regarding  the  "drunk"  in  the 
classical  random  walk  --oblem  can  now  be  restated  in  the 
context  of  the  requisition  processing  system  aa  follows: 

a.  Given  the  transaction  is  in  one  of  the  transient 
states  (first  pass  or  subsequent  pass),  what  is  the 
probability  the  transaction  will  end  up  as  a  “MRO"?;  as 

a  passing  order??  as  a  reject  to  customer??  as  a  backorder? 

b.  Given  the  transaction  is  in  one  of  the  transient 
states  (first  pass  or  subsequent  pass)  how  many  times,  on 
the  average,  will  the  transaction  be  in  the  subsequent  pass 
state  prior  to  ending  up  as  a  MRO?;  as  a  passing  order?:  as 
a  reject  to  customer??  as  a  backorder? 
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D.  Some  Results  with  Model  A 

1.  The  Model  A  of  the  requisition  processing  system 
was  processed  by  the  MARK1  computer  program.  The  computer 
print-out  is  includec  in  a  later  section.  Some  of  the 
results  are  as  follows: 

2.  The  expected  number  of  times  of  remaining  in  a 
transient  state  prior  to  being  absorbed  are  obtained 
from  the  fundamental  matrix.  The  results  are  as  follows: 

a.  A  first  pass  (FrsPAS)  transaction  can  be  expected 
to  be  in  a  fir3t  pass  state  only  once  prior  to  being 
absorbed.  This,  agrees  as  it  should  with  the  " common 
sense"  of  the  system. 

b.  A  subsequent  pass  (SUBPAS)  transaction  can  be 
expected  to  be  in  a  first  pass  state  zero  times  prior  to 
being  absorbed.  This  also  agrees  as  it  should  with  the 
"common  sense"  of  the  system  by  implying  that  once  a 
transaction  is  a  subsequent  pass  transaction  it  cannot  pass 
through  the  first  pass  state. 

c.  A  first  pass  (FRSPAS)  transaction  can  be  expected 
to  be  in  a  subsequent  pass  state  about  . 23  times  before 
being  absorbed. 
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d.  A  subsequent  pass  transaction  can  be  expected  to 
be  in  a  subsequent  pass  state  1.30  times  prior  to  being 
absorbed. 

3.  The  mean  number  of  cycles  until  absorbtion  is 
obtained  from  the  T  Matrix. 

a.  The  mean  number  of  cycles  until  completion  for 
a  first  pass  transaction  is  1.23. 

b.  The  mean  number  of  cycles  until  completion  for 
a  subsequent  pass  transaction  is  1.30. 

4.  The  probability  that  a  transaction  is  completed 
given  it  was  initially  in  a  particular  state  is  obtained 
from  the  U  Matrix. 

a.  The  probability  is  .56  that  a  first  pass  trans¬ 
action  will  become  an  MRO  and  .08  that  a  subsequent  pass 
transaction  will  become  an  MRO. 

b.  The  probability  is  .04  that  a  first  pass  trans¬ 
action  will  become  a  passing  order  and  .12  that  a  subse¬ 
quent  pass  transaction  will  become  a  passing  order. 

c.  The  probability  is  .08  that  a  first  pass  trans¬ 
action  will  become  a  reject  to  customer  and  .29  that  a 
subsequent  pass  transaction  will  become  a  reject  to 
customer. 
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d.  The  probability  is  .31  that  a  first  pass  trans¬ 
action  will  go  on  back-order  and  .52  that  a  subsequent 
pass  transaction  will  go  on  tack-order. 

5.  The  number  of  cycles  required  to  reduce  the 
fraction  of  subsequent  pass  transactions  to  1%  or  less  is 
obtained  from  computation  of  the  AM(K)  vector. 

a.  Assuming  initially  that  20%  of  the  transactions 

to  be  processed  are  subsequent  pass,  it  can  be  expected  to 
take  three  (3)  cycles  for  the  fraction  of  subsequent  pass 
transactions  to  drop  to  1%  or  less.  In  two  (2)  cycles  the 
fraction  of  subsequent  pass  transactions  can  be  expected 
to  drop  to  about  4%. 

b.  The  same  percentages  as  above  essentially  hold 
even  if  initially  100%  of  the  transactions  to  be  processed 
are  subsequent  pass  transactions. 


E.  A  Variation  of  Parameter  Study  on  Model  B. 

1.  initially,  Model  B  was  run  with  transition  Matrix 
values  as  depicted  in  the  earlier  section  showing  the 
transition  matrix  diagram.  The  computer  print-out  is 
included  in  a  later  section. 

2.  A  variation  of  parameter  study  was  conducted  on 
Me'  :1  B.  The  purpose  of  the  study  is  to  develop  charts 
and  graphs  depicting  relationships  between  the  input  and 
output  of  Model  B  over  a  complete  range  in  values  of 
particular  parameters.  The  parameters  involved,  their 
possible  range  in  variation,  and  the  parametric  values 
studied  are  shown  in  the  following  table: 


Parameter 

Possible 

Range 

Parametric 
Values  Studied 

First  Pass 

to 

O.v  to  1.0 

.01. 

.05, 

.10,  .15, 

Subsequent 

Pass 

.20. 

.25. 

.50,  .  75, 

Transition 

Probability 

.99, 

.999 

Subsequent 

Pass  to 

0.0  to  1.0 

.05, 

.10, 

.25,  .50 

Subsequent 

Pass 

Transition 

Probability 
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INPUT: 


Model  B  was  studied  for  three  initial  vector  condi¬ 


tions  . 

100%  first  pass  transactions  and 
0%  subsequent  pass  transactions. 
80%  first  pass  transaction  and 
20%  subsequent  pass  transactions. 
0%  first  pass  transactions,  and 
100%  subsequent  pass  transactions. 
OUTPUT:  Model  B  was  studied  for  the  following  Outputs: 

(1)  The  model  was  studied  for  the  number  of  cycles 
required  to  yield  an  output  vector  such  that  the  percent 
of  subsequent  pass  transactions  (incompleted  transactions) 
is  1%  or  less.  This  is  equivalent  to  the  number  of  cycles 
required  to  yield  an  output  vector  such  that  the  percent 
of  completed  transactions  is  99%  or  more. 

(2)  Also,  the  average  quantity  of  cycles  expected  for 
a  first  pass  transaction  and  a  subsequent  pass  transaction 
to  be  completed  were  studied. 

3.  The  results  are  presented  on  the  following  charts. 
Chart  E-l  summarizes  the  results  obtained  for  100%  first- 
pass  input  and  also  for  the  mixed  input  (80%  first-pass  and 
20%  subsequent  pass).  The  solrd  curves  represent  the  100% 


(1)  (0,  1,  0): 

(2)  (0,  .80,  .20)  : 

(3)  (0,  0,  1): 
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first-pass  input  and  the  broken  curves  represent  the  mixed 
input . 

The  other  input  condition,  100%  subsequent  pass,  is 
shown  on  Chart  E-2. 

The  average  quantity  of  cycles  expected  fcr  a  first- 
pass  transaction  to  be  completed  is  presented  on  Chart 
E-3.  Chart  E-4  shows  the  average  quantity  of  cycles 
expected  for  a  subsequent-pass  transaction  to  be  completed. 

The  basic  data  for  preparing  Charts  E-l  and  E-2  is 
presented  in  a  tabular  format  on  Charts  E-5  thru  E-8. 

Discussion  and  interpretation  of  the  charts  and  other 
findings  is  being  withheld  at  this  time  pending  further 
study  of  the  model. 
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QUANTITY  OF  CYCLES,  M,  REQUIRED  TO  REDUCE 
OUTPUT  SUBSEQUENT-PASS  (INCOMPLETE)  TO  1%  OR  LESS 


CHART  E- 1 


QUANTITY  OF  CYCLES 

T  D_Ay  c  7  /'>rp  t  °  r»A 


REQUIRED  SUCH  THAT  THE  PERCENT  OF  OUTPUT 
SUBSEQUENT— Pass  ui<COKPLETE)  IS  1%  OR  LESS 


TNTTIAL-  CONDITIO”*’  - 


100%  FIRST  PASS  INPUT  (0,  1,  0}  AND  MIXED  INPUT  (0,  80.  .20) 


QUANTITY  OF  CYCLES,  M.  REQUIRED  TO  REDUCE 
OUTPUT  SUBSEQUENT-PASS  (INCOMPLETE)  TO  1%  OR  LESS 


CHART  E-2 


QUANTITY  OF  CYCLES  REQUIRED  SUCH  THAT  THE  PERCENT  OF  OUTPUT 
TRANSACTIONS  FOR  SUBSEQUENT- PASS  (INCOMPLETE)  IS  1%  OR  LESS 

INITIAL  CONDITIONS: 

100*  SUBSEQUENT-PASS  INPUT  (0,  0,  1) 


SUBSEQUENT-PASS  TO  SUBSEQUENT-PASS  TRANSITION  PROBABILITY  VALUE 

20 


AVERAGE  QUANTITY  OF  CYCLES  FOR  A 
FIRST-PASS  TRANSACTION  TO  BE  COMPLETED 


CHART  E-3 


AVERAGE  QUANTITY  OF  CYCLES  EXPECTED  FOR  A 
FIRST-PASS  TRANSACTION  TO  BE  COMPLETED 


SUBPAS 

TO 

SUBPAS 


FIRST-PASS  TO  SUBSEQUENT-PASS  TRANSITION  PROBABILITY  VALUE 

21 
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CHART  E-5 


TABULATION  OF  BASIC  DATA 

SUB-PASS  TO  SUB-PASS  TRANSITION  PROBABILITY  VALUE=  .  05 


1^^ 

FIRST  PASS  TO  SUBSEQUENT  PASS 

TRANSITION  PROBABILITY  VALUES 

HU 

.999 

.99 

.75 

.50 

.25 

.20 

.15 

.  10 

.05 

.01 

INITIAL 

CONDITION  *: 

(0,  1 

.  0) 

i 

.  999 

.990 

.750 

.  500 

.250 

.  200 

.150 

.  100 

19 

.010 

2 

.050 

.049 

.037 

.025 

.012 

.010 

.007 

.005 

ESI 

.000 

3 

.002 

.002 

.  002 

.001 

.001 

1 

4 

wtm 

5 

1 

6 

1 

7 

1 

8 

■ 

INITIAL  CONDITION  *j 

(0,  « 80,  .20) 

I 

.810 

.810 

.610 

.410 

.210 

.170 

.130 

.090 

.050 

.002 

2 

.040 

.040 

.030 

.020 

.010 

.  008 

.007 

.002 

.000 

3 

.002 

.002 

.002 

.001 

.001 

4 

5 

6 

7 

a 

INITIAL 

CONDITION  *i 

(0.  0,  1) 

i 

.050 

.050 

.050 

.050 

.  050 

.  050 

.050 

.050 

o  050 

.050 

2 

3 

.  002 

.002 

.002 

.002 

.002 

.002 

.002 

.002 

.00? 

.002 

4 

5 

6 

7 

a 

•INITIAL  CONDITION  (%  Computed.,  %  FIRST  PASS,  %  SUBSEQUENT  PASS) 
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CHART  E-6 


TABULATION  OP  BASIC  DATA 

SUB-PASS  TO  SUB-PASS  TRANSITION  PROBABILITY  VALUE"  10 


FIRST  PASS  TO  SUBSEQUENT  PASS 
TRANSITION  PROBABILITY  VALUES 


99  .75  .50  .25  .20 


INITIAL  CONDITION  *:  (0,  1,  0) 


.10  .05  .01 


.500 

.250  : 

.200 

.050 

.025 

.020 

.005 

.002  j 

1 

.002 

15  .010 


50  .010 

05  .001 


INITIAL  CONDITION  *:  (0,  .80,  .20) 


20 

.220  | 

.  180 

.140 

42 

.022 

.018 

014 

04 

.002 

: 

INITIAL  CONDITION  (0,  0,  1) 


.  100 

.  100 

100 

.100 

.100 

.  100 

.010 

.010 

.010 

. 

.010 

•  010 

.010 

♦INITIAL  CONDITION  (%  Computed. ,  %  FIRST  PASS,  %  SUBSEQUENT  PASS) 
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CHART  E-7 


TABULATION  Of  BASIC  DATA 

SUB-PASS  TO  SUB-PASS  TRANSITION  PRORARTLITY  VALUE-. 25 


QTY 

CYCLES  , 

FIRST  PASS  TO  SUBSEQUENT  PASS 

TRANSITION  PROBABILITY  VALUES 

(STEPS) 

.999 

.99 

.75 

.50 

.25 

.20 

.15 

.10 

.05 

.01 

INITIAL 

CONDITION  *: 

(0,  1 

.  0) 

1 

.  999 

.990 

.750 

.  500 

.250 

.2  00 

.  150 

.100 

.050 

.010 

2 

.250 

.250 

.  187 

.  125 

.062 

.  050 

.037 

025 

012 

002 

3 

.062 

.062 

.047 

.031 

.016 

.012 

.009 

.  006 

.003 

.001 

4 

.016 

.015 

.012 

.008 

.004 

5 

.004 

.003 

6 

7 

8 

.  _  . 

- 

INITIAL  CONDITION  *: 

(0,  ,80,  .20) 

l 

.850 

•  850 

.650 

.450 

.250 

.210 

.170 

.  130 

.090 

.060 

2 

.210 

.162 

.112 

.050 

.042 

.  032 

.022 

.014 

3 

.053 

.041 

.028 

.013 

.010 

.003 

.006 

.004 

4 

.014 

.012 

.010 

.007 

.004 

5 

.003 

6 

7 

8 

_ 

INITIAL 

CONDITION  *: 

£0,  C 

).  1) 

1 

.250 

.250 

.250 

.250 

.250 

.  250 

.250 

.250 

.250 

.250 

2 

.062 

.062 

.062 

.062 

.062 

.  062 

.062 

,  062 

062 

.  06  2 

3 

.016 

.016 

.016 

.016 

.016 

.  016 

.016 

.016 

.016 

.016 

4 

C 

.004 

.004 

.004 

.004 

.004 

.004 

.004 

.  004 

.004 

.004 

6 

7 

8 

•INITIAL  CONDITION  {%  Computed.,  %  FIRST  PASS,  %  SUBSEQUENT  PASS) 
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CHART  E-8 


TABULATION  OK  BASIC  DATA 

SUB-PASS  TO  SUB-PASS  TRANSITION  PROBABILITY  VALUE=  .50 


FIRST  PASS  TO  SUBSEQUENT  PASS 

1 

QTY 

TRANSITION  PROBABILITY  VALUES 

CYCLES  , 

(STEPS) 

.999 

.99 

.75 

.50 

.20 

.15 

.  10 

.05 

.01 

INITIAL 

CONDITION  *: 

(0,  1 

,  0) 

I 

.  999 

.990 

.750 

.  500 

1 

.250 

.  200 

m 

.  100 

.050 

.010 

2 

.499 

.499 

.  375 

.250 

.125 

.  100 

.  050 

025 

.005 

3 

.250 

.250 

.187 

.  125 

.  062 

.050 

mm 

.  025 

.012 

.002 

4 

.125 

.125 

.  094 

.062 

.031 

.025 

.  019 

.  012 

.001 

5 

.062 

.062 

.  04  7 

.031 

.012 

.009 

.  006 

.003 

.001 

6 

.031 

.031 

.023 

.016 

.006 

.  005 

.  00  3 

.002 

7 

.016 

.015 

.012 

.008 

.003 

.  002 

.  001 

.001 

8 

.008 

.008 

INITIAL  CONDITION  *j 

(0,  .80,  .20) 

1 

.892 

.892 

.  700 

■I 

.300 

.  260 

.220 

.  180 

.140 

2 

.450 

.450 

.  350 

.150 

.  130 

.110 

.090 

.070 

3 

.225 

.225 

.175 

.125 

.075 

.  065 

.055 

.045 

.035 

4 

.112 

.112 

.087 

.062 

.037 

.  032 

.028 

.  022 

.017 

5 

.056 

.056 

.044 

.019 

.016 

.013 

.Oil 

.009 

6 

.028 

.028 

.022 

ISIIIe 

.  010 

.  008 

.007 

.  006 

005 

003 

7 

.014 

.014 

.011 

.008 

.005 

.004 

.003 

.  003 

.001 

a 

.007 

.007 

INITIAL 

CONDITION  *: 

(0,  0,  1) 

1 

.500 

.500 

.  500 

.  500 

.  500 

.  500 

.500 

.  500 

.500 

.500 

2 

.250 

.250 

.250 

.  -5C 

.250 

.250 

.250 

.  250 

.250 

.250 

3 

.125 

.125 

.  125 

.125 

.  125 

.125 

.  125 

.  125 

.125 

.  125 

4 

.062 

.062 

.062 

.062 

.  062 

.  062 

.  062 

.  062 

.062 

.062 

5 

.031 

.031 

.031 

.031 

.031 

.031 

.031 

.  031 

.031 

.031 

6 

.016 

.016 

.016 

.  016 

.016 

.016 

.016 

.016 

7 

8 

u - 

.008 

.008 

.008 

■ 

H 

.008 

.008 

.008 

.008 

.008 

U _ _ _ 1 _ I _ I - 1 - 1 - 1  ..  -  I _ I _ 

♦INITIAL  CONDITION  (*  Computed. ,  %  FIRST  PASS,  %  SUBSEQUENT  PASS) 


F.  Mathematical  Terminology  and  Formulae 

1.  Transition  Probability  Matrix;  A  square  array  of 
numbers  all  of  whose  entries  for  each  row  add  up  to  one. 


1.1 

Example : 

NEXT 

1  STATE 

STATE 

HOME 

2 

3 

4 

5 

6 

7 

BAR 

HOME 

1 

0 

0 

0 

0 

0 

0 

0 

2 

3/4 

0 

1/4 

0 

0 

0 

0 

0 

PRESENT 

3 

0 

3/4 

0 

1/4 

0 

0 

0 

0 

STATE 

4 

0 

0 

3/4 

0 

1/4 

0 

0 

0 

5 

0 

0 

0 

3/4 

0 

1/4 

0 

0 

6 

0 

0 

0 

0 

3/4 

0 

1/4 

0 

7 

0 

0 

0 

0 

0 

3/4 

0 

1/4 

BAR 

0 

0 

0 

0 

0 

0 

0 

l 

1.2 

Explanation : 

The 

entries 

for 

each  particular  row 

represent  the  probabi 

.•  ities 

of  an 

item 

going  to  the 

corres- 

ponding 

column  state 

given 

it  was 

initially  in 

the 

state 

corresponding  to  the  particular  row.  i.e..  The  probability  of 
going  from  state  4  to  state  2  is  0,  to  state  3  is  .750,  and  to 
state  5  is  .250. 

2.  Absorbing  States:  Row  probability  vectors  in  the 
transition  probability  matrix  having  "Is1'  on  the  diagonal  of 
the  matrix  and  all  the  other  entries  are  "Os"  are  referred 
to  as  absorbing  states,  i.e.,  row  "home"  and  row  "bar"  are 
absorbing  states. 
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3.  Transient  States  (non-absorbing)  ■  probability 

vectors  in  the  transition  probability  matrix  not  having  ”ls" 
on  the  diagonal  of  the  matrix  are  referred  to  as  transient 
states,  i.e.,  rows  "2",  "3",  "4",  and  "S'*  are  transient 


states . 


4 .  Canonical  Form  of  Transition  probability  Matrix: 

When  in  the  transition  probability  matrix,  all  the  absorbing 
states  (rows)  are  grouped  together  at  the  top  of  the  matrix 


with  all  the  "Is"  composing  an  identity  matrix  and  all  of 


the  transient  states  (rows)  are  together  at  the  bottom  of 
the  matrix,  then,  the  transition  probability  matrix  is 
said  to  be  in  canonical  form. 


4 . 1  Example : 


STATE 

H0ME 

BAR  l 

C0RNR2 

C0RNR3 

C0RNR4 

C0RNR5 

C0RNR6 

C0RNR7 

H3ME 

1  .000 

0.000  J 

0  .000 

0.000 

0.000 

0  .000 

0.000 

0.000 

BAR 

0.000 

1 .000  1 

0.000 

0.000 

0.000 

0.000 

0.000 

0  .000 

C  JR  (JR?. 

"  .750 

o.ooo  r 0.01)0 

.250 

0,000 

0.000 

0.000 

o7ooo" 

C0RNR3 

0.000 

0.000  1 

.750 

0  .000 

.250 

0.000 

0.000 

0.000 

C0RNR4 

0.000 

0  .000  , 

0.000 

.750 

0.000 

.250 

0.000 

0.000 

C0RNR5 

0.000 

0.000 

0.000 

*>.000 

.750 

0.000 

.250 

0.000 

C0R  NR  S 

0.000 

0.000 

0.000 

u  .000 

0.000 

.750 

0.000 

.250 

C3RNR7 

0.000 

.250 

0.000 

0.000 

0.000 

0„000 

.750 

0.000 

5. 

Partioninq  Canonical 

Matrix 

:  T’np 

canonical  matrix 

is  subdivided  as  follows: 


C  = 


/  Cl 

i 

i 

1 

C0  \ 

1 

^CR 

i 

i 

i 

i 

CQ  J 
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5.1  Cl:  Identity  Matrix:  The  number  of  rows  equals 
the  number  of  columns  equals  the  number  of  absorbing  states. 
5.11  Example : 


Cl 


STATE  H0ME  BAR 
HOME  1,000  0,000 

BAR  0,000  1.000 


5.2  C0:  Zero  Matrix:  A  matrix  containing  all  zeros 
and  having  the  number  of  rows  equal  to  the  number  of 
absorbing  states  and  the  number  of  columns  equal  to  the 
number  of  transient  states. 

5.21  Example : 

/ 

0  0  0  0  0 

C0  =  1 

U  0  0  0  0  0 

5 . 3  CR:  Matrix  of  Transient  to  Absorbing  Probabilities 
The  entries  represent  the  probability  of  going  from  tran¬ 
sient  state  to  absorbing  state.  The  number  of  rows  equals 
the  number  of  transient  states  and  the  number  of  columns 


equals  the  number  of  absorbing  states. 


5.31  Example : 


CR= 


STATE  H0ME 
C0RNR2  .750 
C0RNR3  0,000 
C0RNR4  0,000 
C0RNR5  0,000 
C0RNRS  0.000 
C0RNR7  0.000 


BAR 

0.000 

0,000 

0,000 

0,000 

0.000 

.250 
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The  entries  represent  the  probability  of 


state  to  transient  state.  The  number  of  rows  equals  the 


number  of  columns  equals  the  number  of  transient  states. 
5.41  Example : 


STATE 

C0RNR2 

C0RNRS 

C0RNR4 

C0RNR5 

C0RNR6 

C0RNR7 

C3RNR2 

0.000 

.250 

0.000 

0.000 

0.000 

0.000 

C0RNR3 

.750 

0.000 

.250 

0.000 

0  .000 

0.000 

C0RNR4 

0.000 

.750 

0.000 

.250 

0.000 

0.000 

C3R  NR  5 

0.000 

0.000 

.750 

0.000 

.250 

0.000 

C0RNR6 

0.000 

0.000 

0.000 

.750 

0.000 

,250 

C3RNR7 

0.000 

0.000 

0.000 

0.000 

.750 

0,000 

6.  FN:  Fundamental  Matrix:  Each  entry,  FN  (I.J),  is 
the  expected  number  of  times  in  state  J  (column)  before 
being  absorbed,  given  that  the  present  state  is  I  (row) . 
The  number  of  rows  equals  the  number  of  columns  eguals  the 


number  of  transient  states. 


FN  =  (NI-CO) 


NOTE:  NI  is  an  identity  ma  rix,  established  for  the  compu¬ 

tation  of  FN.  NI  has  the  number  of  rows  equal  to  the 
number  of  columns  equal  to  the  number  of  transient  states. 


6.1  Computation  of  FN:  FN  is  computed,  using  the 
following  power  series  approximation: 

FN  -  NI  +  CQ  +  (CQ)2  +  (CQ)3  +  _ 


6 . 2  Example ; 


;TATE 

C0RNR2 

C0RNR3 

C0RNR4 

C0RNR5 

C0R  NR  6 

C0RNR7 

C0RNR2 

1  .332 

.443 

•  1  46 

.048 

.015 

.004 

C0RNR3 

1  .328 

1  4  771 

.586 

.190 

.059 

.015 

C0R  NR  4 

i  .317 

1  ,757 

1  .903 

.618 

.190 

.048 

C0RNR5 

1  .284 

1  .713 

1  .855 

1  .903 

.586 

.146 

C0RNR6 

1 .18$ 

1  .581 

1  .713 

1  .757 

1  .771 

.443 

C0RNR7 

.889 

1 .186 

1  .284 

1  .317 

1  .328 

1  .332 

30 


7. 


T: 


Matrix  of  Absorption  Times;  Each  entry  is  the 


mean  time  to  absorption  (number  of  states  passed  through, 
including  final  state  and  not  including  initial  state,  in. 
order  to  be  absorbed).  This  matrix  is  a  column  vector  with 
the  number  of  rows  equal  to  the  number  of  transient ' states. 

7.1  Computation  of  T;  Each  entry  of  T  is  equal  to  the 
row  sum  of  each  row  of  FN.  This  is  accomplished  by  estab¬ 
lishing  a  column  vector,  01  of  l's  having  the  same  number 
of  rows  equal  to  the  number  of  transient  states. 

T  =  (FN)  •  (01) 

7. 2  Example : 

STATE 

C0RNR2  1.987 
C0RNR3  3.949 

T  C0RNR4  5.833 
C3RNR5  7.487 
C3RNRS  8.450 
^  0R  NR  7  7,337 

8.  U;  Matrix  of  Absorption  Probabilities;  Each  entry 
is  the  probability  of  being  absorbed,  given  it  was  initially 
in  a  transient  state.  The  number  of  rows  equals  the  number 
of  transient  states,  and  the  number  of  columns  equals  the 
number  of  absorbing  states. 

8. 1  Computation  of  U; 

U  =  (FN)  •  (CR) 
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8 . 2  Example : 


STATE 

H0ME 

BAR 

C0RNR2 

.999 

.001 

C3RNR3 

.996 

.004 

C0RNR4 

.9RB 

.012 

C0RNR5 

.963 

.037 

C0RNR6 

.889 

.11  1 

C0RNR7 

.567 

.333 

9.  CQMP !  Matrix  of  Transition  Probabilities  for  M 


Steps ;  Each  entry  is  the  probability  of  going  from  state 
to  state.  The  number  of  rows  equals  the  number  of  columns 
equals  the  number  f  absorbing  and  transient  (non-absorbing) 


states . 

9. 1  Computation  of  CQMP: 

CQMP  =  (C)M 

9. 2  Example : 


For  M  =  3 


CQMP  * 


STATE 

H0ME 

BAR 

C0R NR 2 

C0RNR3 

C0RNR4 

C0RNR5 

C0RNR6 

C0RNR7 

H3ME 

1  .000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

BAR 

0.000 

1  .000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

C0RNR2 

.891 

0.000 

0.000 

.094 

0.000 

.016 

0.000 

0.000 

C0RNR3 

.362 

0.000 

.281 

0.000 

.141 

0.000 

.016 

0.000 

C0RNR4 

.422 

0.000 

0.000 

.422 

0.000 

.141 

0.000 

.016 

C0RNR5 

0.000 

.016 

.422 

0.900 

.422 

0.000 

.141 

0.000 

C0RNR6 

0.000 

.062 

0.000 

.422 

0.000 

.422 

0.000 

.094 

C0RNR7 

0.000 

.297 

0.000 

0.000 

.422 

0.000 

.281 

0.000 
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10.  A;  Initial  state  Spare  Probability'  Rev  Vector; 
initial  values  for  each  of  the  possible  states  (columns) . 

The  number  of  rows  is  one,  and  the  number  of  columns  equals 
the  number  of  absorbing  and  transient  states 

10.1  Example : 

A  -  (0,  0,  1/  ,  1/6,  1/6,  1/6,  1/6,  1/6) 

11.  AM:  State  Space  M  Steps  Later;  Each  entry  is  the 
value  for  each  of  the  states,  M  steps  later,  given  initial 
values  A. 

11.1  Computation  of  AM: 

AM  =  (A)  •  (CQMP) 

11.2  Example: 

For  M-3, 

STATE  H0ME  BAR  C0RNR2  C0RNR3  C0RNR4  C0RNR5  C0RNR6  C0RNR7 

.313  .063  .117  .157  .164  .097  .073  .018 

12.  Summary:  For  absorbing  Markov  chains,  the  following 
three  questions  are  usually  of  interest: 

a.  What  is  the  probability  that  the  process  will  end 
up  in  a  given  absorbing  state? 

Answer:  Entries  in  Matrix  U. 
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b.  Oil  tuc  Svei Layer  iiuw  luiiy  will  it  Lake  for  the 
process  to  be  absorbed? 

Answer:  Entries  in  Matrix  T. 

c.  On  the  average,  how  many  times  will  the  process 
be  in  each  transient  (non-absorbing)  state? 

Answer:  Entries  in  Matrix  FN. 
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u . 


riow  cnart  (Fortran  Program  -  MARKl/2/3) 
Simulation  with  Markov 
Transition  Matrix  Model  ot 
A  Requisition  Processing  System 


MARK1 


100* 


100/130 


220 


230 


230/240 


T,  A ,  AM,  STATL 
L  i  I,  S 


♦Program  Line  Numbers 


t 
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COMPUTE.. 
CGMir,J)=0 
llll.j)  --o 


con  pc  i,  j) 

19 1,8;  J-,,8 


CONTROL-  1 


500 


580/590 


t;  20 


690/720 


730 


820/850 


© 

_L 


pr  in: 

CAT,  J 
I  A, KU; 

JAlU  _ 

r 

con  pl 
KIU^I 
LIU  *1 

he; 
AESOP 
APS  Ok 

Print:  Transition 

Matrix,  C,  in 
Canonical  Form. 


PRiNT. 

|  Ml,LL) 
A/'/,  KIU 


COMPUTE.  . 

KRL  ^  TAliSOp+'l 
t\RU  -KU 
LRU  -  Tp.P.  WH 
K  QU  --  KRU~K  PLrl 


Print:  Identity  Matrix. 


Print:  Matrix  of 

transient  to 
absorbing 
probabilities . 


COMPUTE. : 

KQL  »ZABSOR+l. 
kqu=ku 

LQL  -  TAESORn 


PRINT: 
\C{KR,  LR) 

KR  *KRL.  KPU 
\LRALRU '  — 


38 


COMPUTE. : 
caMfk’a.La^curiia, 
L£)  'tCSXKO,  PbfC^Ul\ 


MARK2 


Transient 
probability 
Matrix  for  M 
steps. 


0  for  same 
Vector  A, 
1  for  new 
Vector  A. 


Qty  of  elements, 
K,  in  Vector  A. 


5 


CONPd !  r.  ■ 

AMij)  -  A/'Uj)  f- 
I)  *  (SI  M PC  £  ,j-)j 

KU 


PRINT 
ASUS) 
J-  l,i.H 


© 


Print:  Vector,  AM, 

for  State  Space 
M  steps  Later. 


Input:  MX 

0  to  end 
Computation, 
1  to  Do  next 
set  data, 

S-fr-1  to  skip  s 
sets  of  data 
and  do  next 
set  of  data. 


ru\ixi\J 


4000 


4010 


4020/4030 


4060 


Obtains  Max  and  Min 
Entry  in  Array  X. 


xm N  -  Y 


I 


ii.  COMPUTER  FORTRAN  PROGRAM  MARK  l/.'/J 

R  K  | 


ioo  print  "simulation  with" 

110  PRiNT  "-V5RK0V  TRANSITION  MATRIX  MOTEL  BF" 

IV'  PRINT  "A  REQUISITION  PROCESSING  SYSTIM" 

1  '  7  1R1NT  *3Y  IRWIN  GOODMAN" 

MO  PRINT  'INPUT,  C,  TRANSITION  MATRIX  IN  CANONICAl  FORM- 
150  PRINT  "IN  LINES  1420  THRU  1300  BY  ROWS* 

160  PR' NT  "PRECEDE  C  MATRIX  DATA  WITH  NAMES  BE  STATES" 

170  PRINT  'RIGHT  JUSTIFY  BETWEEN  COMMAS  WITH  SPACFS  ADl'H)  FOR" 

1 30  PRINT  “7  CHARACTER  FORMAT." 

IRQ  PRINT  *  FXAMPLF  •  HOME,  BAR,  t;  OR  NR2  ,F  TC  .  F0U.0WU'  "Y  LA  T/ 

220  CONTROL  =  1 

2  JO  DI  MENS  I  ON  C(  8 , 8)  ,  CQ  "  ,  CQM"  ,  CQMP"  ,  CQMP  1  "  ,  Nl  '  ,  FN  "  .  U*  ,  X  "  ,  T  <  « )  ,  A  "  .  ‘  >  ' 
2 AO  DI MENSI ON  STATE(g) 

250  PRINT  “INPUT  QTY  OF  ABSORBING  STATES =  1ABS0R " 

2 SO  INPUT,  IABS0R 

270  PRINT  "INPUT  QTY  0E  TRANSIENT  STATES =1 TRA NS " 

280  INPUT,  1TRANS 

2 9 0 C  COMPUTE  QTY  BF  ROWS  AND  COLUMNS  IN  MATRIX  C, 

*00  K  U  -I  ABS0R  +  ITRANS 
510  LU=IABS0R  +  ITRANS 

320  PRINT  "QTY  0F  ROWS  IN  MATRIX  C  :',KU 

330  PRINT  'QTY  BF  COLUMNS  IN  MATRIX  C  :",UI 

3 AO  1  D0  65  I =1  ,8 

350  TCI 3  =0*  65  AMU)  =0 

360  D0  66  1=1 ,8 

370  D0  66  J=! ,8 

380  CQ  M(I  ,J )  =Oj  U<I,J)=0J  66  CQ  MP ( I , J ) =0 

390  G0  TO  (3,4),  CONTROL 

400  3  READ,  ( STA TE < J ) , J = I , LU) 

410  4 

420  READ,  C  CCC I  ,J)  ,J  =  1  ,LU)  ,1=1  ,XII) 

4  50  PRINT 
440  PRINT 

450  PRINT  "TRANSITION  MATRIX,  C,  IN  CANONICAL  FORM" 

460  PRINT 

470  PRINT  BB,  "STATE  ", (STATE (J) ,J = 1 , LU) 

480  BB  :  FORMAT ( I  0A7) 

490  DO  B ,  I =1 ,KU 

500  Bj  PRINT  AA,  STATEC I ) , CC (I , J ) , J= I , LU) 

510  AA:  F0RMAKA7,  10F7.3) 

520  PRINT 
530  PRINT 

540  PRINT  "PARTITION  TRANSITION  MATRIX  (CANONICAL  FORM)" 

550  PRINT  "INTO  FOLLOWING  HATRIC1ES:" 

560  PRINT  "Cl:  IDENTITY  MATRIX" 

570  PRINT 
580  KI UrIABSOR 
640  LI  1MABS0R 

50 n  PRINT  RB. 'STATE  " , (STATE (LI > , LI = I , LI U ) 

610  DO  D,  V 
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H.  COMPUTER  FORTRAN  PROGRAM  MARK  1/2/3  (CONTD) 


MARK  1  C0NTINUED 


620  Di  PRINT  AA.STATECKI)  ,<C(KI  , LI)  ,LI  =  1  , LIU) 
i  6 30  PRINT 

:<  .  640  PRINT 

650  PRINT  “CR:  MATRIX  0F  TRANSIENT  T0  ABS0RBING  PR0BABI LITIES-* 
6S0  PRINT  "PR0BABIL1TY  0F  G0ING  FR 0M  TRANSIENT  STATE” 

{  670  PRINT  ”T0  ABS0RBI  NG  STATE** 

|  680  PRINT 

i  65 0  KRL  =  IASS0R  +  I 

700  KRU=KU 

;  710  LRUrlABSflR 

720  KRN=KRU-KRL  +  1 

730  PRINT  BB,  "STATE  ",  (STATE(LR) ,LR  = 1 ,LRU) 

740  D0  E,  KR=KRL,KRU 

750  Ej  PRINT  AA,  STATE(KR),  <C(XR , LR> ,LR=1 ,LRU> 

760  PRINT 
770  PRINT 

780  PRINT  "CQj  MATRIX  0F  TRANSIENT  T0  TRANSIENT  PROBABILITIES-” 
790  PRINT  'PROBABILITY  0F  GOING  FROM  TRANSIENT  STATE” 

■  800  PRINT  ”T0  TRANSIENT  STATE" 

810  PRINT 
820  KGL=IABS0R  +  1 
!  830  KQ LisKU 

f  840  LQI=IAES0R  +  1 

850  LQU=LU 

860  KQNsKQU  -  KQL  +  1 
870  LQN=LQU  -  LQL  +  1 

880  PRINT  B3,  STATE  ,  (STATE(LQ) ,  LQ=LGL,LQU) 

890  D0  Ff  KQ=KQL,KQU 

900  F:  PRINT  AA,  STA TECXQ ) , <C(KQ ,LQ ) ,LQ=L9L ,LQU) 

910  PRINT 
920  PRINT 

930  PRINT  "C0MPUTATI0N  0F  THE  FUNDAMENTAL  MATRIX" 

940C  ESTABLISH  IDENTITY  MATRIX,  NI ,  F0R  C0MPUTATI0N  0F 
I  950C  FUNDAMENTAL  MATRIX,  FN. 

k  960  D0  40  KfisKQL.KSU 

!  970  D0  30  LQsLQL.lfiU 

980  IF  (Kfi-LQ)  20,10,20 
i  990  10:  NIOCQ,LQ)=l 

}  1000  G0  T0  30 

i  1C10  20:  NKKQ.L8)  =0 

i  1020  30:  C0NTINUE 

!  1030  40:  C0NTINUE 

i  .  1040  PRINT  "FN:  FUNDAMENTAL  MATRIX-" 

'  1050  PRINT  "EACH  ELEMENT  IS  THE  EXPECTED  NUMBER  0F  TIMES” 

1  1060  PRINT  "IN  STATE  JCC0LUMN)  BEF0RE  BEING  ABS0RBED" 

’  1070  PRINT  "GIVEN  THAT  THE  PRESENT  STATE  IS  I (R0W) " 

3  1080  PRINT 

1090  PRINT  BB,  "STATE  ",  (STATE(LQ) ,L9=L«L,LQU) 

1100  D0  50  Kfl  =  X(JL.KQU 
UlO  D0  50  LQsUL.LQU 
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K.  COMPUTER  FORTRAN  PROGRAM  MARK  1/2/3  (CONTD, 
■HARK!  rflNTI  UIJED 


1120  FN(KQ  ,LG)  "NI  <KQ,LQ)  +  CCKQ ,LQ) 

1130  CQ(K9,LQ>=C(KQ,LQ> 

1140  50  C0NTINIJE 

1150C  COMPUTE  THE  FUNDAMENTAL  MATRIX,  FN,  EQUAL  T0  THE  INVERSE 
USOC  0F  FNI=NI-CQ,  BY  SERIES  APPR0XIMATI0N 
1  1  70  70 

1180  D0  <»0  KQ=KQL,KQU 
1190  D0  90  LG=LQL,LQU 
1200  CQM(KQ,LQ)*0 
1210  D 0  SO  K  =KQL  KQ U 

1220  CQM(KQ,LQ>=CQM(KQ,LQ)+CQ(KQ,X>*C(K,LQ) 

1230  80  CONTINUE 

1240  90  C0NTINUE 

1250  D0  120  KQ=KQL,KQU 

1260  D0  120  LQ=LQL,LQU 

12  70  FN(KQ , LQ ) =FN(KQ , LQ )+CQM<  KQ ,LQ  ) 

1280  120  CONTINUE 

1290  CALL  AMXMN(CQM,KQL,KQU,LQL,LQU,AX,BX> 

1300  IF  (ABS(AX)-.OOOl)  140,140,150 

1310  140  IF<ABS(BX)-„000U  160,160,150 

1320  150  D0  500  KQrKQL.KQU 

1330  D0  500  LQsLQL.LQU 

1340  500  CQ(KQ,L9)=CQM(KQ,LQ) 

1350  G0  T0  70 

1360  150  D0  G,KQ=K9L,KQU 

1370  G:  PRINT  AA,  STATE(KQ),  (FN (KQ ,LQ) ,LQ =LQL ,LQU> 

1380  PRINT 
1390  PRINT 
1400  $USE  MARK2 

1410  JDATA 

1411  H0ME ,  BAR,  C0RN82,  C0RNR3,  C0RNR4 ,  C0RNR5 ,  C0RNR6 ,  C0RNR7 
14121 .,7*0. 

1413  0.,1 .,6*0. 

1414  .75, 2*0. ,.25, 4*0. 

141  5  2*0  . ,.75,0. ,.25, 3*0  . 

141  5  3*0.  ,.75,0. ,.25, 2*0  . 

141  7  4*0.,  .75,0., .25,0. 

1 41  8  5*0 ., .75,0 ., .25,0 . , .25,4*0. , .75,0. 

1419  C0MPLT,  FT?  SPAS,  SUBPAS 

1440  l.,0.,0. 

1441  .99,0., .01 

1442  .75,0., .25 

1450  1 . ,0 . ,0. 

1451  .95,0., .05 

1452  .75,0., .25 

1460  1  . ,0.  ,0  . 

1461  .90  ,0., .10 

1462  .75,0.,. 25 

1470  1.  0.,0. 

1471  .85,0. ,.15 
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U.  CGMF'JTEn  rufciKAN  PROGRAM  MARK  1/2/3  (CONTD) 
MARK1  CONTINUED 


1472 

1460 

1481 

1482 

1490 

1491 

1492 

1500 

1501 

1502 

1510 

1511 

1512 

1520 

1521 

1522 

1530 

1531 

1532 
1801 
1802 
1803 


75,0 
. ,  0  . 
80,0 
75  0 
.  ,0  . 
75,0 
75  0 
.  ,0 , 
50,0 
75  0 
. ,  0 , 
25,0 
75,0 
« ,0  , 
01 ,0 
75,0 
,0. 


,.25 

0. 

,  .20 
,.25 
6. 

..25 

a:25 

,.50 

,.25 

0. 

..75 

,.25 

0. 

,.99 

..25 

0. 


>001  ,0  . , .999 
,75,0. ,.25 

W?0,  PAS0RD,  REJCUS 
« , 5*0  . , 0 . , 1 « ,4*0. ,2*0. 
.55, .02, .03, .22,0. ,.18, 


,  BACK0R,  FRSPAS ,  SUBPAS 
,1., 3*0., 3*0.,!., 2*0. 
.06, .09, .22, .40,0. ,.23 


MARK2 


2000  PRINT  "COMPUTATION  OF  THE  MATRIX  0F  ABSORPTION  TIMES" 

2010  PRINT  "Tt  MATRIX  0F  ABS0RPTI0N  TIMES-’ 

2020  PRINT  "EACH  ELEMENT  IS  THE  MEAN  TIME  T0  ABS0RPTI0N" 

2010  PRINT  "(NUMBER  0F  STATES  PASSED  THROUGH ,  INCLUDING  FINAL  STATE" 
2040  FRINT  "AND  N0T  INCLUDING  INITIAL  STATE,  IN0RDER  T0  BE  ABSORBED) " 
2050  PRINT 

2060C  COMPUTE  T  =  FN  TIMES  C  WHERE  C  IS  A  C0LUMN  VECTOR  0F  I  's. 

2070C  THE  ELEMENTS  IN  MATRIX  T  ARE  EQUAL  T0  THE  ROW  SUM  FOR  EACH 
2080C  ROW  OF  FN. 

2090  PRINT  B8,  STATE  " 

2100  D0  180  KQ=KQL,KQU 
2110  DO  1 70  LQ=LQL,LQU 
2120  T(KQ)=T(KQ)  +  FN(KQ.LQ) 

2130  170  CONTINUE 

2140  PRINT  AA,  STATE(KQ) ,T<KQ) 

2150  180  CONTINUE 

21  SO  PRINT 
2170  PRINT 
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H.  COMPUTER  FORTRAN  PROGRAM  MARK  1/2/3  (CONTD) 


MARK2  CONTINUED 


2180  PRINT  “COMPUTATION  0F  THE  MATRIX  0F  ABSORPTION  PROBABILITIES" 
2190  PRINT  "U.  MATRIX  0F  ABSORPTION  PROBABILITIES' 

2200  PRINT  "PROBABILITY  0F  BEING  ABS0RBED , “ 

2210  PRINT  “GIVEN  IT  WAS  INITIALLY  IN  A  TRANSIENT  STATE” 

2220  PRINT 

2230C  MULTIPLY  MATRIX  FN  TIMES  CR . 

2240  D0  190  KQ=KQL,KQU 
2250  DO  190  LR=1,LRU 
22 SO  D0  190  K=KR£,fKRU 

2270  U(KQ ,LR>=U(KQ,LR>  +  FN(KQ ,K)*C(K,LR) 

2280  190  CONTINUE 

2290  PRINT  B8,  “STATE  <STATE(LR) ,LR=I ,IRU> 

2300  D0  GG,  KQ =KQL  XQU 

2310  GG:  PRINT  AA,  STATE(KQ)  ,  (U  (KQ.LR)  ,LR  =  1  ,lRU> 

2320  215 
2330  PRINT 
2340  PRINT 

2350  PRINT  “C0MPUTATI0N  0F  TRANSIENT  PR0BABILITY  MATRIX  F3R  M  STEPS" 
2360  PRINT  “INPUT  NUMBER  0F  STEPS,  M,  “ 

2370  PRINT  'THAT  MATRIX  OF  TRA NSI TI0N  PR0BABI LI TI ES“ 

2380  PRINT  “SH0ULD  BE  C0MPUTED  F0R“ 

2390  rn i ::t  “  input  ri=o  if  C0mputati0n  N0T  desired” 

2400  INPUT,  M 
2410  IF(M-O)  300.300,220 
2420C  G0  T0  END  FOR  M=0 
2430  220  PRINT 
2440  PRINT 

2450  PRINT  “CMP:  fttTRIX  0F  TRANSITI0N* 

2460  PRINT  “PR08ABI LI TI ES  F0R  M  STEPS, Ms", M 

2470  PRINT  'PR0BABILITY  0F  G0ING  FR0M  STATE  T0  STATE" 

2480  PRINT 

2490C  COMPUTE  C  T0  THE  M  P0WER. 

2500  D0  225  Isl  ,XU 
2510  D0  225  Jrl.LU 
2520  225  CfiMPl (I ,  J ) =  C  ( I ,  J ) 

2530  D0  260  L=2,M 
2540  T0  250  1=1  ,KU 
2550  D0  250  J=1 ,LU 
2560  CQMPU  ,J)=0 
2570  D0  230  K=! ,KU 

2580  CQMPCI , J) =C8MP ( I ,  J) +CQ MP 1 (I ,K>*C(K,J> 

2590  230  C0NTINUE 

2600  250  C0NTINUE 

2610  D0  255  1=1  ,KU 

2620  D0  255  J=1 ,LU 

2630  255  C8MP1  (I  ,J)  =CQMP(I  ,J) 

2640  260  CONTINUE 

2650  PRINT  BB,  “STATE  (STATE CJ> ,J= 1 ,LU> 

2660  D0  H , I  =1  ,Kl’ 

2670  Ht  PRINT  AA ,  STATE(I),  (CQMP(I ,J> ,J=1 ,LU> 
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H.  COMrUTER  FORTRAN  PROGRAM  mark  1/2/2  (CCNTD) 
MARK2  CONTINUED 


2680  PRI NT 
2690  PRINT 

2700  PRINT  "COMPUTATION  0F  STATE  SPACE,  M  STEPS  LATER," 

2710  PRINT  “GIVEN  INITIAL  STATE' 

2720  PRINT  "INPUT  ISAMEA=0  IF  SAME  R0W  VECTOR  TO  BE  USED" 

2730  PRINT  "INPUT  ISAMEAsI  F0R  NEW  AOO  VECTOR  INPUT." 

2740  INPUT, ISAMEA 

2750  IF(ISAMEA-l)  270,265,270 

2760  265 

2770  PRINT  INPUT  INITIAL  STATE  AS  A  R0W  VECTOR  IN  FOLLOWING  F0RM" 
2780  PRINT"K:“,KU 

2790  PRINT  "INPUT  K=0  IF  COMPUTATION  NOT  DESIRED" 

2800  PRINT  “INPUT  K,A(1 ) ,A<2)  ,A(3) . ACK)" 

2810  INPUT, K,<A(J),J=1,K) 

2820  IFCK-O)  215,215,270 
2830C  REPEAT  M  STEP  COMPUTATION, 

2840  270  PRINT 
2850  PRINT 

2860  PRINT  “AMt  ROW  VECTOR  FOR  STATE  SPACE  M  STEPS  LATER, M=",M 
2870  PRINT  “GIVEN  INITIAL  STATE  A" 

2880C  COMPUTE  AM=A  TIMES  CQMP. 

£890  rRINT 

2900  D0  275  J;1,LU 

2910  275  AM(J)=0 

2920  DO  290  J=1 ,LU 

2930  D0  290  1=1, XU 

2940  AM<J)=AM<J>+A<n+CQMP(1,J) 

2950  290  CONTINUE 

2960  PRINT  BB,  "STATE  ",  (STATE(J) ,J=1 ,LU> 

2970  PRINT  CC,  (AMCJ)  ,J  =  I  ,LU> 

2980  CCs  FORMATOX,  10F7.3) 

2990  GO  T0  215 

3000  300  PRINT  "INPUT  MX=0  TO  END  COMPUTATION" 

3010  PRINT  "MX=1  TO  D0  NEXT  SET  OF  DATA" 

3020  PRINT  " MX=S+1  TO  SKIP  S  SETS  OF  DATA" 

3030  PRINT  'AND  DO  NEXT  SET  OF  DATA" 

3040  INPUT, MX 
3050  IFCMX-1)  330,2,310 
3060  310  MXU=MX-1 
3070  D0  320  K= 1 ,MXU 

3080  320  READ, ( (C<I , J> ,J =1 , LU> , 1=1 ,KU> 

3090  2  CONTROL  =  2 
3100  GO  TO  1 
3110  330  END 
3120  $USE  MARK3 
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H.  COMPUTER  FORTRAN  PROGRAM  MARK  1/2/3  l CONTD) 


MARK  3 


4000  FMBR0UTINE  AMXMN<X, I L , I U, JL ,JU, AX, BX> 
40  10  DI MENS!  0N  X(8.5) 

4020  XMIN  =  l.E20 
4030  XMAX -~1  .£-20 
4040  D3  400  X :I L, IU 
4050  D0  400  L=JL,JU 
40  SO  Y=X(K,LJ 

4070  IF(Y-XMIN)  420,420,425 

4080  420  X MI N  =Y 

40R0  425  C0NTINUE 

4100  I F (Y-XKAX)  400,400,525 

4110  525  XMAX=Y 

4120  400  CONTINUE 

4130  AX=XMAX 

4140  BX=XMIN 

4150  RETURN 

4150  END 
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I.  COMPUTER  TIME  SHARING  TERMINAL  PRINT -OUTS 
I. I  Classical  Random  Walk 


MARK  1  1  3t  1 1  MAN. - 07/08/68 

IN  MARK2 
IN  .FIRST 
IN  MARK3 
IN  .FIRST 

SIMULATION  WITH 

MARK0V  TRANSI TI0N  MATRIX  M0DEL  0F 
A  REQU1SITI0N  PROCESSING  SYSTEM 
BY  IRWIN  G00DMAN 

INPUT  C,  TRANSI TI0N  MATRIX  IN  CAN0NICAL  FARM 

IN  LINES  1420  THRU  1800  BY  R0WS 

PRECEDE  C  MATRIX  DATA  WITH  NAMES  0F  STATES 

RIGHT  JUSTIFY  BETWEEN  COMMAS  WITH  SPACES  ADDED  F0R 

7  CHARACTER  FORMAT. 

EXAMPLE.  H0ME,  BAR,  C0RNR2 ,ETC.  F0LL0WED  BY  DATA 
INPUT  QTY  0F  ABS0RBING  STATES  =IAB50R 

7  72 

INPUT  QTY  0F  TRANSIENT  STATES =1  TRANS 
7  76 

QTY  0F  ROWS  IN  MATRIX  C  =  8 

QTY  0F  COLUMNS  IN  MATRIX  C  =  8 


TRANSITION  MATRIX,  C,  IN  CANONICAL  F0RM 


STATE 

HOME 

BAR 

C0RNR2 

C0RNR3 

C0RNR4 

COR  NR  5 

C0RNR6 

C0RNR7 

HOME 

1  .000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

BAR 

0.000 

i  .000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

C  OR  NR2 

.750 

0.000 

0.000 

.250 

0.000 

0.000 

0.000 

0.000 

COR  NR 3 

0.000 

0.000 

.750 

0.000 

.250 

0.000 

0.000 

0.000 

COR  NR 4 

0.000 

0.000 

0.000 

.750 

0.000 

.250 

0.000 

0.000 

COR  NR  5 

0.000 

0.000 

0.000 

0.000 

.750 

0.000 

.250 

0.000 

C  OR  NR  S 

0.000 

0.000 

0.000 

0.000 

0.000 

.750 

0.000 

.250 

CORNR7 

0.000 

.250 

0.000 

0.000 

0.000 

0.000 

.750 

0.000 

PARTITION  TRANSITION  MATRIX  (CANONICAL  F0RM) 
INTO  FOLLOWING  WATRICIESj 
Cl:  IDENTITY  MATRIX 

STATE  HOME  BAR 
HOME  1  .000  0  .000 

BAR  0.000  1.000 
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I.  COMPUTER  TIME  SHARING  TERMINAL  PRINT-OUTS  (CONTD) 


1.1  Classical  Random  walk 


CR:  MATRIX  0F  TRANSIENT  T0  ABSORBING  PR0RARTLITIES- 
FR  OBABi  LI  I T  Bh  GOING  FROM  TRANSIENT  STATE 
T0  ABS0RBI NG  STATE 


!  STATE 

H0ME 

bAR 

;  C0RNR2 

.750 

0.000 

COR  NR 3 

0.000 

0.000 

C0RNR4 

0.000 

0.000 

C0RNR5 

0.000 

0.000 

C0RNR5 

0.000 

0.000 

C0RNR7 

0.000 

.250 

CQ:  MATRIX  0F  TRANSIENT  TO  TRANSIENT  PR0BABI LI TI ES- 
PR0BABILITY  0F  G0I  NG  FR0M  TRANSIENT  STATE 
T 0  TRANSIENT  STATE 


STATE 

C0RNR2 

COR  NR  3 

C0RNR4 

C0RNR5 

C0RNR6 

C0RNR7 

C0RNR2 

0.000 

.250 

0.000 

0.000 

0.000 

0.000 

C0RMR3 

.750 

0.000 

.250 

0.000 

0.000 

0.000 

C0RNR4 

0.000 

.750 

0.000 

.250 

0.000 

0.000 

C0RNR5 

0.000 

0.000 

.750 

0.000 

.250 

0.000 

C0RNR6 

0.000 

0.000 

0.000 

.750 

0.000 

.250 

C0RNR7 

0.000 

0.000 

0.000 

0.000 

.750 

0.000 

C0MPUTATI 0N  0F  THE  FUNDAMENTAL  MATRIX 
FN:  FUNDAMENTAL  MATRIX- 

EACH  ELEMENT  IS  THE  EXPECTED  NUMBER  0F  TIMES 
IN  STATE  J (COLUMN)  BEF0RE  BEING  ABS0R8ED 
GIVEN  THAT  THE  PRESENT  STATE  IS  I (ROW) 


STATE 

C0RNR2 

C0RNR3 

COR  NR  4 

C0RNR5 

COR  NR  6 

C0RNR7 

C0RNR2 

1  .332 

.443 

.146 

.048 

.015 

.004 

COR  NR  3 

1  ,  32,8 

1.771 

.586 

.190 

.059 

.015 

C0R NR  4 

1  .317 

1 .757 

1  .903 

.618 

.190 

.048 

C0RNR5 

1  .284 

1  .713 

1  .355 

1  .903 

.586 

.146 

COR  NR  6 

1  .18* 

1 .581 

1  .713 

1  .757 

1  .771 

.443 

C0RNR7 

.089 

1 .186 

1  .284 

1  .317 

1  .328 

1  .332 

COMPUTATION  0F 

THE  MATRIX  0F 

ABSORPTION  TIMES 

T:  MATRIX  0F  ABS0RPTI0N  TIMES- 

EACH  ELEMENT  IS  THE  MEAN  TIME  T0  ABS0RPTI0N 

(NUMBER  0F  STATES  PASSED  THR0UGH,  INCLUDING  FINAL  STATE 

AND  N0T  INCLUDING  INITIAL  STATE,  IN0RDER  T0  BE  ABS0RBED) 


STATE 


C0RNR2 

1  .987 

C0RNR3 

3.949 

C  0R  NR  4 

5.833 

COR  NR 5 

7.487 

C0RNR6 

8.450 

COR  NR  7 

7.337 

l£. 


I.  COMPUTER  T T M >•'  «Hnp.l;:c  TCKMI ri/Uj  i'jKI NT— OUTS  (CO  TD) 


1.1  Classical  Random  Wallc 

COMPUTATION  0F  THE  MATRIX  OF  ABSORPTION  PROBABILITIES 
U.  MATRIX  0F  ABSORPTION  PROBABILITIES 
PR0BABILITY  0F  BEING  ABSORBED. 

GIVEN  IT  WAS  INITIALLY  IN  A  TRANSIENT  STATE 


STATE  HOME  BAR 


C0RNR2 

.999 

.001 

COR  NR 3 

.996 

.004 

COR  NR 4 

.988 

.012 

C0RNN5 

.9  S3 

.037 

COR  NR  6 

.889 

.Ml 

C0RNR7 

.667 

.333 

COMPUTATION  OF  TRANSIENT  PROBABILITY  MATRIX  FOR  M  STEPS 
INPUT  NUMBER  OF  STEPS,  M, 

THAT  rWTRIX  OF  TRANSITION  PROBABILITIES 
SHOULD  BE  COMPUTED  FOR 
INPUT  M=0  IF  COMPUTATION  NOT  DESIRED 
?  ?2 


CMP:  fWTRIX  OF  TRANSITION 
PROBABILITIES  FOR  M  STEPS,M=  2 

PROBABILITY  OF  GOING  FROM  STATE  TO  STATE 


STATE 

HOME 

BAR 

C  OR  NR  2 

COR  NR  3 

COR  NR  4 

C0RNR5 

CORNRS 

COR  NR  7 

HOME 

1  .000 

0.000 

0.000 

0.000 

0,000 

0.000 

0.000 

0.000 

BAR 

0.000 

1  .000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

C0RNR2 

.750 

0.000 

.187 

0.000 

.062 

0  .000 

0.000 

0.000 

C  OR  NR  3 

.562 

0.000 

0.000 

.375 

0.000 

.062 

0.000 

0.000 

COR  NR  4 

0.000 

0.000 

.552 

0.000 

.375 

0.000 

.062 

0.000 

C  OR  NR  5 

0.000 

0.000 

0.000 

.562 

0.000 

.375 

0.000 

.062 

COR  NR  6 

0.000 

.062 

0.000 

0.000 

.562 

o.noo 

.375 

0.000 

C  OR  NR  7 

0.000 

.250 

0.000 

0.000 

0  .000 

.562 

0.000 

.187 

COMPUTATION  OF 

STATE 

SPACE,  M 

STEPS 

LATER, 

GIVEN  INITIAL  STATE 

INPUT  I SA MEA  =0  IF  SAME  ROW  VECTOR  TO  BE  USED 
INPUT  I SA  MEA : 1  FOR  NEW  A  <  K)  VECTOR  INPUT. 

1 

?  ?i 

INPUT  INITIAL  STATE  AS  A  ROW  VECTOR  IN  FOLLOWING  FORM 
K  -  6 

INPUT  K=0  IF  COMPUTATION  NOT  DESIRED 

INPUT  K,A(1)  ,A<2) ,A<3) . ACK) 

7  78, 0,0  ,.  I  67,  .167  ,.16?,.  I  67,  .167,.!  67 


AM.  ROW  VECTOR  FOR  STATE  SPACE  M  STEPS  LA  TER ,M=  2 

GIVEN  INITIAL  STATE  A 

STATE  HOME  BAR  C0RNR2  COR  NR  3  C  OR  NR  4  COR  NR  5  C0RNR6  C0RNR7 

.219  .052  .125  .157  .157  .167  .073  .042 
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1.  COMPUTER  'T'lME  SHARING  TERMINAL  PRIOT-OUTS  fCONTM 
t  1  CLASS  I  C.V.  RANDOM  wAjujs 

r  r*  MP  UTAH  ON  OF  TRANSIENT  PROBABILITY  MATRIX  F0R  M  STEPS 
INPUT  NUMBER  0F  STEPS,  M, 

THAT  MATRIX  OF  iMfiNSITION  PROBABILITIES 
SHOULD  BE  COMPUTED  FOR 
INPUT  M;0  IF  COMPUTATION  NOT  DESIRED 
?  725 


CMP;  MATRIX  OF  TRANSITION 
PROBABILITIES  FOR  M  STEPS,  «=  25 

PROBABILITY  OF  GOING  FROM  STATE  TO  STATE 


STATE 

HOME 

BAR 

C0RNR2 

COR  NR  3 

COR  NR 4 

COR  NR 5 

CORNRS 

C0RNR7 

HOME 

1  .000 

0  .000 

0.000 

0.000 

0.000 

0.000 

0.000 

0,000 

BAR 

o.noo 

1  .000 

0  .000 

0.000 

0.000 

0.000 

o.noo 

0.000 

COR  NR2 

.999 

.001 

0.000 

.000 

0.000 

.000 

0.000 

.000 

C0RNR3 

.995 

.004 

.001 

0.000 

.001 

0  .  ii  0  0 

.000 

0.000 

COR  NR 4 

.986 

.012 

0  .000 

.002 

0.000 

.001 

0.000 

.000 

COR  NR  5 

.959 

.037 

.003 

0.000 

.002 

0.000 

.001 

0.000 

CORNRS 

.884 

.111 

0  .000 

.004 

0.000 

.002 

0.000 

.000 

CORNR7 

.660 

.333 

.003 

0.000 

.003 

0.000 

.001 

0.000 

COMPUTATION  OF  STATE  SPACE,  M  STEPS  LATER, 
GIVEN  INITIAL  STATE 

INPUT  ISAMEA=0  IF  SAME  ROW  VECTOR  TO  BE  USED 
INPUT  I  SA  MEA  =  I  FOR  NEW  A  ( K)  VECTOR  INPUT. 

?  70 


AM.  ROW  VECTOR  FOR  STATE  SPACE  M  STEPS  LATER, M=  25 

GIVEN  INITIAL  STATE  A 

STATE  HOME  BAR  COR  NR  2  C  OR  NR  5  COR  NR  4  COR  NR  5  C0RNRS  C0RNR7 
.916  .083  .001  .001  .001  .000  .000  .000 


COMPUTATION  OF  TRANSIENT  PROBABILITY  MATRIX  FOR  M  STEPS 
INPUT  NUMBER  OF  STEPS,  M, 

THAT  MATRIX  OF  TRANSITION  PROBABILITIES 
SHOULD  BE  COMPUTED  FOR 
I N ° U T  M^O  IF  COMPUTATION  NOT  DESIRED 
?  73 


CMP:  MA TRI X  OF  TRANSITION 
PROBABILITIES  FOR  M  STEPS, M=  3 

PROBABILITY  OF  GOING  FROM  STATE  TO  STATE 


STATE 

HOME 

BAR 

COR  NR 2 

C0RNR3 

C0RNR4 

C^RNR5 

C0RNR6 

C0RNR7 

HOME 

1  .000 

0.000 

0.000 

0.000 

0,000 

U  .000 

0.000 

0.000 

BAR 

0.000 

1  .000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

C0RNR2 

.891 

0.000 

0.000 

.094 

0.000 

.016 

0.000 

0.000 

C0RNR3 

.562 

0.000 

.281 

0.000 

.141 

0.000 

.016 

0.000 

C0RNR4 

.422 

0.000 

0.000 

.422 

0.000 

.141 

0.000 

.016 

C0RNR5 

0.000 

.016 

.422 

0.000 

.422 

0.000 

.141 

0.000 

C  0R  NR  6 

0.000 

.0  62 

0.000 

.422 

0.000 

.422 

0.000 

.094 

C0RNR7 

0.000 

.297 

0.000 

0.000 

.422 

0.000 

.281 

0.000 
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COMPUTER  TIME  SHARING  TERMINAL  PRINT-OUTS  (CONTD) 

T.l  rr.ARRTr-aT.  DUrnnu 


I  . 


COMPUTATION  0F  STATE  SPACE, 


M  STEPS  LATER, 


INPUT  ISAMEA=0  IF  SAME  ROW  VECT0R  T0  BE  USED 
INPUT  ISAM£A  =  !  F0R  NEW  A(K)  VECTOR  INPUT. 

?  70 


AM.  R0W  VECTOR  F0R  STATE  SPACE  M  STEPS  LATER, M=  3 

GIVEN  INITIAL  STATE  A 

STATE  H0ME  BAR  C0RNR2  C0RNR3  COR  NR 4  COR NR  5  C0RNR6  C0RNR7 

.313  .063  .117  .157  .164  .097  .073  .018 


COMPUTATION  0F  TRANSIENT  PR0BABI LI TY  MATRIX  F0R  M  STEPS 
INPUT  NUMBER  0F  STEPS,  M, 

THAT  MATRIX  0F  TRANSITION  PR0BABI  LI  TIES 
SH0ULD  BE  COMPUTED  F0R 
INPUT  M=0  IF  COMPUTATION  NOT  DESIRED 
?  75 


CMP:  MATRIX  0F  TRANSITION 
PROBABILITIES  F0R  M  STEPS, Ms  5 

PROBABILITY  OF  GOING  FROM  STATE  TO  STATE 


STATE 

HOME 

BAR 

C0RNR2 

COR  NR 3 

C0RNR4 

C0RNR5 

C0RNR6 

COR  NR  7 

HOME 

1  .000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

BAR 

0.000 

1  .000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

COR  NR 2 

.943 

0.000 

0.000 

.044 

0.000 

.012 

0.000 

,001 

COR  NR  3 

.773 

.001 

.132 

0.000 

.079 

0.000 

.015 

0.000 

COR  NR  4 

.659 

.004 

0.000 

.237 

0.000 

.088 

0.000 

.012 

COR  NR 5 

.316 

.024 

.316 

0.000 

.264 

0.000 

.079 

0.000 

COR  NR  6 

.237 

.086 

0.000 

.396 

0.000 

.237 

0.000 

.044! 

COR  NR  7 

0.000 

.314 

.237 

0.000 

.315 

0.000 

.132 

0.000 

COMPUTATION  OF 

STATE  : 

SPACE,  i 

M  STEPS 

LATER, 

GIVEN  INITIAL  STATE 

INPUT  I SA MEA  =0  IF  SAME  ROW  VECTOR  TO  BE  USED 
INPUT  I SA  MEA : 1  FOR  NEW  A (K)  VECTOR  INPUT. 

7  70 


AM.  R0W  VECTOR  FOR  STATE  SPACE  M  STEPS  LATER, M=  5 

GIVEN  INITIAL  STATE  A 

STATE  HOME  BAR  C0RNR2  COR NR 3  COR  NR 4  COR NR 5  CORNR6  C0RNR7 

.489  .072  .114  .113  .110  .056  .038  .009 


COMPUTATION  CF  TRANSIENT  PROBABILITY  MATRIX  FOR  M  STEPS 
INPUT  NUMBER  OF  STEPS,  M, 

THAT  WTRIX  OF  TRANSITION  PROBABILITIES 
SHOULD  BE  COMPUTED  FOR 
INPUT  M=0  IF  C0MPUTATI  ON  NOT  DESIRED 
7  70 
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1.2  Model  A  of  Requisition  Processing  System 


MARK1  1  3 j  45  M0N.-~ 07/08/68 

I  M  >: 

'•  ri  .  i-  !  R '  ; 

if, 


SIMULATION  WITH 

MARKOV  TRANSITION  MATRIX  MODEL  0F 
A  REQUISITION  PROCESSING  SYSTEM 
BY  IRWIN  GOODMAN 

INPUT,  C,  TRANSITION  MATRIX  IN  CANONICAL  FORM 

IN  LINES  1420  THRU  1800  BY  ROWS 

PRECEDE  C  MATRIX  DATA  WITH  NAMES  0F  STATES 

RIGHT  JUSTIFY  BETWEEN  COMMAS  WITH  SPACES  ADDED  FOR 

7  CHARACTER  FORMAT. 

EXAMPLE.  HOME,  BAR,  C0RNR2,ETC.  FOLLOWED  BY  DATA 
INPUT  QTY  OF  ABSORBING  STATES=IABSOR 
?  ?4 

INPUT  QTY  OF  TRANSIENT  STATES: I  TRANS 
?  ?2 

QTY  OF  ROWS  IN  MATRIX  C  :  6 

QTY  OF  COLUMNS  IN  MATRIX  C  =  6 


TRANSITION  fttTRIX,  C,  IN  CANONICAL  FORM 


STATE 

WO 

PASORD 

REJCUS 

BACK OR 

FRSPAS 

SUBPAS 

WO 

1  .000 

0.000 

0,000 

0.000 

0.000 

0.000 

PASORD 

0.000 

1  .000 

0.000 

0.000 

0.000 

0.000 

REJCUS 

0.000 

0.000 

1  .000 

0.000 

0.000 

0.000 

BACK  OR 

0.000 

0.000 

0.000 

1  .000 

0.000 

0.000 

FRSPAS 

.550 

.020 

.030 

.220 

0.000 

.180 

.  JBPAS 

.OSO 

.090 

.220 

.400 

0.000 

.230 

PARTITION  TRANSITION  MATRIX  (CANONICAL  FORM) 
INTO  FOLLOWING  MATRICIESj 
Cl:  IDENTITY  MATRIX 


STATE 

MRO 

PAS0RD 
REJCUS 
BACK  OR 


MRO 
1  .000 
0.000 
0.000 
0.000 


PAS0RD 
0,000 
1 .000 
0.000 
0.000 


REJCUS 
0.000 
0.000 
1  .000 
0.000 


BACK OR 
0.000 
0.000 
0.000 
1  .000 
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1.2  i4odel  A  of  Requisition  Processina  System 


Ck  t  mTRIX  0F  TRANSIENT  TO  ABSORBING  PROB  AB I  LI  TI ES- 
PR  SCAD  I  LI  Ti  or  uojnu  rnion  in«  nsi  tm  SlHlt 
T0  ABSORBING  STATE 

STATE  ffi0  PASORD  REJCUS  BACKOR 
FRSPAS  .550  .020  .030  .220 

SUBPAS  .060  .090  .220  .400 


CQt  mTRIX  0F  TRANSIENT  T0  TRANSIENT  PR08ABILI  TI  ES- 
PR0BABILITY  0F  G0ING  FROM  TRANSIENT  STATE 
T0  TRANSIENT  STATE 

STATE  FRSPAS  SUBPAS 
FRSPAS  0.000  .180 

SUBPAS  0.000  .230 


COMPUTATION  0F  THE  FUNDAMENTAL  MATRIX 
FN-  FUNDAMENTAL  MATRIX- 

EACH  ELEMENT  IS  T"E  EXPECTED  NUMBER  0F  TIMES 
IN  STATE  JCC0LUMN)  BEFORE  BEING  ABSORBED 
GIVEN  THAT  THE  PRESENT  STATE  IS  I  (ROW) 

STATE  FRSPAS  SUBPAS 
FRSPAS  1  .000  ,234 

SUBPAS  0.000  1.299 


COMPUTATION  0F  THE  MATRIX  0F  ABSORPTION  TIMES 

T:  MATRIX  0F  ABSORPTION  TIMES- 

EACH  ELEMENT  IS  THE  MEAN  TIME  T0  ABSORPTION 

(NUMBER  0F  STATES  PASSED  THROUGH ,  INCLUDING  FINAL  STATE 

AND  NOT  INCLUDING  INITIAL  STATE,  INORDER  T0  BE  ABSORBED) 

STATE 

FRSPAS  1.234 
SUBPAS  1.299 


COMPUTATION  OF  THE  MATRIX  OF  ABS0RPTI0N  PR0BABI LITIES 
U  MATRIX  0F  ABSORPTION  PROBABILITIES 
PROBABILITY  0F  BEING  ABSORBED, 

GIVEN  IT  WAS  INITIALLY  IN  A  TRANSIENT  STATE 

STATE  MR0  PACORD  REJCUS  BACK0R 

FRSPAS  .564  .041  .081  .31  4 

SUBPAS  .078  .1  17  .286  .519 


COMPUTATION  OF  TRANSIENT  PROBABILITY  MATRIX  FOR  M  STEPS 
INPUT  NUABER  OF  STEPS,  M 
THAT  MATRIX  0F  TRANSITION  PROBABILITIES 
SHOULD  BE  COMPUTED  FOR 
INPUT  M=0  IF  COMPUTATION  NOT  DESIRED 
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I  .  2  Mode  1  A  of 

CMP:  fttTRlX  OF  TRANSITION 
PR0BABI LI  TI  ES  FOR  M  STEPS. M= 


'Requisition  Procesnina  Svs(-*i» 


j  i  n  i  l.  IV  Jinit 


STATE 

MRO 

PAS3RD 

REJCUS 

BACK0R 

FRSPAS 

SUBPAS 

MRO 

1  .000 

0  .000 

0.000 

0.000 

0.000 

O.OCO 

PAS0RD 

0.000 

1  .000 

0 .000 

0,000 

0.000 

0.000 

REJCUS 

0  .000 

0.000 

l  .000 

0.000 

0  .000 

0.000 

BACKOR 

0.000 

0  .000 

0.000 

1  ,000 

0.000 

0.000 

FRSPAS 

.561 

.036 

.070 

.292 

0.000 

.041 

SUBPAS 

.074 

.111 

.271 

.492 

0.000 

.053 

COMPUTATION  OF 

STATE 

SPACE, 

M  STEPS 

LATER, 

GIVEN  INITIAL  STATE 

INPUT  ISAMEA=0  IF  SAME  ROW  VECT0R  TO  BE  USED 
INPUT  I SA MEA ; 1  F0R  NEW  A ( K)  VECTOR  INPUT. 

?  71 

INPUT  INITIAL  STATE  AS  A  R0W  VECT0R  IN  F0LL0WI N6  F0RM 
K=  6 

INPUT  K=0  IF  C0MPUTATI0N  N0T  DESIRED 
INPUT  K.AC1) ,A<2) ,A(3) ,...,A<K) 

7  76,0,0  ,0,0, .SO, .20 


AM.  R0W  VECTOR  F0R  STATE  SPACE  M  STEPS  LATER, M:  2 

GIVEN  INITIAL  STATE  A 

STATE  MR0  PAS0RD  REJCUS  BACK0R  FRSPAS  SUBPAS 

.463  .051  .110  .332  0.000  .044 


COMPUTATION  0F  TRANSIENT  PR0BABI LITY  MATRIX  FOR  M  STEPS 
INPUT  NUMBER  OF  STEPS,  M, 

THAT  MATRIX  OF  TRANSITION  PROBABILITIES 
SHOULD  BE  COMPUTED  FOR 
INPUT  M=0  IF  COMPUTATION  NOT  DESIRED 
7  73 


CMP t  MATRIX  OF  TRANSITION 
PROBABILITIES  FOR  M  STEPS, M  =  3 

PROBABILITY  OF  GOING  FROM  STATE  T0  STATE 


STATE 

MRO 

PASORD 

REJCUS 

BACKOR 

FRSPAS 

SUBPAS 

MRO 

1  .000 

0.000 

0.000 

0.000 

0,000 

0.000 

PASORD 

0.000 

1  .000 

0.000 

0.000 

0.000 

0.000 

REJCUS 

0.000 

0.000 

1  .000 

0.000 

0.000 

0.000 

BACKOR 

0  .000 

0.000 

0.000 

1 .000 

0.000 

0.000 

FRSPAS 

.563 

.040 

.079 

.309 

0.000 

.010 

SUBPAS 

.077 

.115 

.282 

.513 

0.000 

.012 

AM.  ROW  VECTOR  FOR  STATE  SPACE  M  STEPS  LATER, M= 
GlOEN  INITIAL  STATE  A 


STATE  PRO  PAS0RD  REJCUS  BACK0R  FRSPAS  SUBPAS 

.466  .055  .119  ,349  0,000  .010 
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1  i  M'-'del  q  riucessmg  System 


MAFK1  14:20  MON  .---0  7/08/68 

IN  MARK2 
IN  .FIRST 
IN  MARK3 
IN  .FIRST 

SIMULATION  WITH 

MARKOV  TRANSITION  MATRIX  MODEL  OF 
A  REQUISITION  PR0CESSI NQ  SYSTEM 
BY  IRWIN  G00DMAN 

INPUT,  C,  TRANSITI0N  MATRIX  IN  CA N0NICAL  FORM 

IN  LINES  1420  THRU  1800  BY  R0WS 

PRECEDE  C  MATRIX  DATA  WITH  NAMES  0F  STATES 

RIGHT  JUSTIFY  BETWEEN  C0MMAS  WITH  SPACES  ADDED  FOR 

7  CHARACTER  FORMAT. 

EXAMPLE,  H0ME,  BAR,  C0RNR2,ETC.  F0LL0WED  BY  DATA 
INPUT  QTY  0F  ABS0RBING  STATES=IABS0R 
?  71 

INPUT  QTY  0F  TRANSIENT  STATES=ITRANS 
7  72 

QTY  0F  R0WS  IN  MATRIX  C  :  3 

QTY  0F  C0LUMNS  IN  MATRIX  C  :  3 


TRANSITI0N  MATRIX ,  C,  IN  CAN0NICAL  F0RM 

STATE  COfPLT  FRSPAS  SU3PAS 
C0MPLT  1.000  0.000  0.000 

FRSPAS  .320  0.000  .180 

SUBPAS  .770  0.000  .230 


PARTITION  TRANSITION  MATRIX  (CANONICAL  FORM) 
I NT0  FOLLOWING  MATRICIESt 
Cl:  IDENTITY  MATRIX 

STATE  C0MPLT 
COMPLT  1.000 


CR:  MATRIX  OF  TRANSIENT  TO  ABSORBING  PR0BABI LI TIES- 
PF08ABILITY  0F  GOING  FR0M  TRANSIENT  STATE 
TO  ABSORBING  STATE 

STATE  COMPLT 
FRSPAS  .820 
SUBPAS  .770 
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1.3  Model  B  of  Requisition  Processing  System  (contd) 


CQj  MATRIX  0F  TRANSIENT  TO  TRANSIENT  PROBABILITIES- 
PROBABILITY  OF  GOING  FROM  TRANSIENT  STATE 
T0  TRANSIENT  STATE 

STATE  FR SPA S  SUBPAS 
FRSPAS  0.000  .180 

SUBPAS  0.000  .?30 


COMPUTATION  OF  THE  FUNDAMENTAL  MATRIX 
FN.  FUNDAMENTAL  MATRIX- 

EACH  ELEMENT  IS  THE  EXPECTED  NUMBER  OF  TIMES 
IN  STATE  J (COLUMN)  BEFORE  BEING  ABSORBED 
GIVEN  THAT  THE  PRESENT  STATE  IS  I  (ROW) 

STATE  FRSPAS  SUBPAS 
FRSPAS  1  .000  .234 

SUBPAS  0.000  1  .299 


COMPUTATION  OF  THE  MATRIX  OF  ABSORPTION  TIMES 

Ts  MATRIX  OF  ABSORPTION  TIMES- 

EACH  ELEMENT  IS  THE  MEAN  TIME  TO  ABSORPTION 

(NUtOER  OF  STATES  PASSED  THROUGH ,  INCLUDING  FINAL  STATE 

AND  NOT  INCLUDING  INITIAL  STATE,  INORDER  TO  BE  ABSORBED) 

STATE 

FRSPAS  1.234 
SUBPAS  1.299 


COMPUTATION  OF  THE  MATRIX  OF  ABSORPTION  PROBABILITIES 
U.  MATRIX  OF  ABSORPTION  PROBABILITIES 
PROBABILITY  OF  BEING  ABSORBED. 

GIVEN  IT  WAS  INITIALLY  IN  A  TRANSIENT  STATE 

STATE  COMPLT 
FRSPAS  1.000 
SUBPAS  1.000 


COMPUTATION  OF  TRANSIENT  PROBABILITY  MATRIX  FOR  M  STEPS 
INPUT  NUMBER  OF  STEPS,  M, 

THAT  MATRIX  OF  TRANSITION  PROBABILITIES 
SHOULD  BE  COMPUTED  FOR 
INPUT  M=0  IF  COMPUTATION  NOT  DESIRED 
?  ?2 


CMP  l  MATRIX  OF  TRANSITION 
PROBABILITIES  FOR  M  STEPS, Ms  2 

PROBABILITY  0ir  GOING  FROM  STATE  TO  STATE 

STATE  COMPLT  FRSPAS  SUBPAS 
COMPLT  1.000  0.000  0.000 

FRSPAS  ,959  0.000  .041 

SUBPAS  .947  0.000  .053 
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Model  B  of  Requisition  Processing  System  (contd) 


1 .  3 


C0MPUTATI0N  0F  STATE  SPACE,  M  STEPS  LATER, 

GIVEN  INITIAL  STATE 

INPUT  ISAMEArO  1 »-  SAME  ROW  VEC10H  10  BE  USED 
INPUT  ISAMEAri  F0R  NEW  A(K>  VECT0R  INPUT. 

7  ?1 

INPUT  INITIAL  STATE  AS  A  R0W  VECT0R  IN  F0LL0WI NG  F0RM 
K=  3 

INPUT  K  =  0  IF  COMPUTATION  N0T  DESIRED 
INPUT  K ,  A  ( 1 ) #A(2) ,  A  ( 3) , ...  ,A(K) 

?  73,0, .80, .20 


AM.  R0W  VECT0R  F0J?  STATE  SPACE  M  STEPS  LATER, M=  2 

GIVEN  INITIAL  STATE  A 

STATE  C0MPLT  FRSPAS  SUBPAS 
.956  0.000  .044 


C0MPUTATI0N  0F  TRANSIENT  PR0BA8ILI TY  MATRIX  F0R  M  STEPS 
INPUT  NUMBER  0F  STEPS,  M, 

THAT  MATRIX  0F  TRANSITI0N  PR0BABILITIES 
SH0ULD  BE  C0MPUTED  F0R 
INPUT  M=0  IF  COMPUTATION  N0T  DESIRED 
?  ?3 


CMP:  MATRIX  0F  TRANSITION 
PROBABILITIES  F0R  M  STEPS , Ms  3 

PROBABILITY  0F  GOING  FR0M  STATE  T0  STATE 

STATE  C0MPLT  FRSPAS  SUBPAS 
C0MPLT  1  .000  0.000  0  .000 

FRSPAS  .990  0.000  .010 

SUBPAS  .938  0.000  .012 


COMPUTATION  0F  STATE  SPACE,  W  STEPS  LATER, 
GIVEN  INITIAL  STATE 

INPUT  ISA MEA=0  IF  SAME  ROW  VECTOR  T0  BE  USED 
INPUT  ISAMEA  =  1  F0R  NEW  A  <K)  VECT0R  INPUT. 
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AM.  F0W  VECT0R  FOR  STATE  SPACE  M  STEPS  LATER, M=  3 

GIVEN  INITIAL  STATE  A 

STATE  C0MPLT  FRSPAS  SUBPAS 
.990  0.000  .010 
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In-ho  ;se 


<1  ABSTRACT 


A  cursory  review  of  the  literature  re'ating  to  the  application  of 
t  e  Markov  Transition  Probability  Matrix  for  t’e  eval  ation  and 
analysis  of  problems  was  accomplished.  A  FORTRAN  IV  computer  time- 
s  arinq  pronram,  based  pon  the  mathematics  of  Markov  Transition 
Matrices,  ’-as  been  developed  and  documented.  The  proqram  was 
initially  developed  with  data  based  pon  a  classical  random  walk 
problem  involvinq  a  dr  ink  meanderinq  from  corner  to  corner  between 
his  home  and  a  bar.  The  resulting  Markov  Model  has  been  applied  to 
a  requisitionina  system,  an  essentially  equivalent  problem.  Some 
anaysis  results  are  presented  followinn  t'e  application  of  t^e 
comp  ter  proqram  to  a  requisitioning  system.  The  computer  program 
has  been  written  generally  enough  for  application  to  sue1,  other 
diverse  problem  areas  as  charge  accounts,  tank  battles,  and  relia- 
1  ility  and  maintainability.  ' 
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